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Abstract 

In this document I recapitulate some results by Hiriart-Urruty and 
Ye[l] concerning the properties of differentiability and the existence of 
directional derivatives of the multiple eigenvalues of a complex Hermitian 
matrix function of several real variables, where the eigenvalues are sup¬ 
posed in a decreasing order. Another version of these results was obtained 
by Ji-guang Sun[5],[6]. 
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1 Differentiability of the eigenvalues of a com¬ 
plex Hermitian matrix 

We will denote by A(C') the spectrum or set of eigenvalues of any complex square 
matrix C. Let C be an open subset of and let A : C —>■ be a matrix 

function of class such that for every x G C the matrix A{x) is Hermitian, i.e. 
A{x)* = A{x) where * denotes the conjugate transpose. As it is well known the 
eigenvalues of A{x) are real numbers; thus, there exist n real functions defined 
on fl, Ai,..., Xn, such that for all x G XI, 

Ai(x) > A2 (x) > • • • > A„(x) 
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are the eigenvalues of A{x). Let m G {1, • ■ • ,n}; it is easy to prove that the 
function Am: fl —?> K is continuous. When the eigenvalue Xm{xo) of A{xq) is 
simple, the function Am is differentiable at xq G But in case of Am(a^o) is a 
multiple eigenvalue of A(xq), Am can be nondifferentiable at xq- For example [3], 
let ^ 

Xi 1x2 
-1x2 -xi. 


A(xi,X2') ■■ = 


be for {xi,X 2 ) G It is obvious that for each {xi,X 2 ) 

A(xi,X 2 ) is Hermitian. Then 


the matrix 


A — Xi —1x2 
1x2 A + xi 


\2 2 2 

= A -x^-X2; 


hence the eigenvalues of A{xi,X 2 ) are ii/xf + Observe that the matrix 

^0 0^ 


> 1 ( 0 , 0 ) = 


0 0 


has a double eigenvalue; but neither the function Ai(a;i,X 2 ) = \/x\ + , nor 

the function A 2 (a;i,a; 2 ) = —\/x\ + x\ are differentiable at (0,0). 

Let d G be a unitary vector, i.e. ||d ||2 = 1, where 11-112 denotes the 
Euclidean norm. The directional derivative of the function Am at the point Xq 
with respect to d is defined as the limit 


Am(a;o,d) 


Am(a^o +td) 
t-^o+ t 


Am (^ 0 ) 


whenever this limit exists. 

In [1, Theorem 4.5] was proved the next theorem. 

Theorem 1 For all xq G II, for all unitary vector d G and for all m G 
{1,..., n}, there exists always 


X'^{xo,d). 

Moreover, it can be proved that X'^{xo, d) is equal to a determined eigenvalue 
of a matrix constructed from A{xif) and d in the following way: For each xq G O, 
there is a unitary matrix U = [ui,..., it„] such that 

U*A{xq)U = diag (Ai(xo),..., A„(a;o)). 

Suppose that Xm(xo) is a multiple eigenvalue of A(a:o), of multiplicity re¬ 
introduce two integers Zm > l;Jm > 0 to precise the position that Xm(xo) 
occupies among the repeated eigenvalues that are equal to it. Consider the 
detailed arrangement of the eigenvalues of >l(a;o): 

-^1 (^0) ^ ‘ ^ Xe—i^ (^0) ^ Xe—i^ + l (^o) = * * * = Am (^o) 

~ Xm-irl {xq) = * * * = Xe-\-jm. (^0) ^ Xe-\-jm + r (^o) ^ ‘ ‘ ‘ ^ -^n (^o) 

That is to say, jm is the number of eigenvalues placed after the subscript m that 
are equal to Xmixo); whereas Zm is the number of eigenvalues placed before m 
that are equal to Xm(xo), plus one (we put Xm(xo) in this list). Hence, jm niay 
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be zero, Jm > !> and im + jm = fm- When m = 1, i.e. if we are considering 
Ai(a;o), we have ii = l,ji = ri — 1. When m = n, i.e. for A„(xo), we have 
in = Tnjjn — 0. In case Am(a^o) is a simple eigenvalue, im = ^,jm = 0. Although 
the notation does not indicate it, the numbers im,jm and Vm depend on xq. 

Let us call t /2 the n X Cm matrix formed by the (m—Zm + l)th, ..., (TO+jm)th 
columns of the matrix U : 


U2 ■ — [aZm — 2^ + 1; ■ • ■ ; ] ! 


i.e. U 2 is formed by Vm orthonormal eigenvectors associated with the eigenvalue 
i^m(xo) of A(xo). For each j € {1,... ,p} define 


dA 

dxj 



aik{x) being the entries of A{x). We will call F'{d) to the x matrix 


F'id) := U* 




U 2 


for every unitary vector d = {di,. ■ ■, dp) G Given that 

/ duik \ _ duik _ dttki 
\ dxj ) dxj dxj ’ 

we have that the matrix is Hermitian; hence. 


dA 


F'(dr = t/| M| U2 


0=1 


= t/j |E‘'« 

0=1 


dA 


U 2 


= *^2 I U2 = F\d)- 


therefore, the matrix F'{d) is Hermitian. Thus, the eigenvalues of F'{d) are real 
numbers. In [1, Theorem 4.5] is proved the following theorem. 

Theorem 2 The directional derivative X'mixojd) is given by 

XUxo,d)=ti^UF\d)) 

where fii^{F'{d)) is the Zmth eigenvalue of F'{d) when the eigenvalues are ar¬ 
ranged in a decreasing order: 


Ml {F\d))>--->yir^ {F'{d)). 


In [I, Corollary 4.3] it is proved the next result. 
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Theorem 3 The function 

^m(^) ■” + “t” ‘ ‘ ‘ “t” “1” * ’ * “1“ X G ^ 

is differentiable at xq . 

From this theorem we can deduce the next corollary. 

Corollary 4 There exists a neighborhood V ofxo, V G fl, in which the function 
tm{x) := Xm-i„^ + lix) + • • • + Xm(x) + • • • + Xm+j^(x) 
is differentiable. 

Proof. Let V C be a neighborhood of xq, sufficiently small so that the 
inequalities 

Xm—ijn(.X^ ^ \m—i^ + l{xf (x) > (x) 

hold when x G V. Let xi be any point of V. Then in the arrangement 

(a^i) ^ * * * ^ (xl) 

of the eigenvalues of A{xi) may have groups of equalities. In view of Theo¬ 
rem 3, the sum of the functions Xi corresponding to each one of these groups, 
is differentiable at Xi; therefore, as tm is the sum of these sums, we have that 
tm is differentiable at xi. I 

2 Differentiability of the singular values of a com¬ 
plex matrix 

Let A: n —>■ be a matrix function of class C^. For each x G C let 

CTi (x) > • • ■ > <Jq (x), with q := min(m, n), 

be the singular values of the matrix Al(x) ordered in a decreasing sense. Thus, 
we can define q functions Ui: fl —>■ M, i G {1,..., g}. We are going to establish 
the properties of differentiability of these functions. By Wielandt’s lemma, the 
m n eigenvalues of the Hermitian matrix 

M(x) := e 

are 

O’! (x) > • ■ ■ > (Tq (x) > 0 = • • • = 0 > —aq (x) > • • ' > —CTl (x) 

(it may have repeated intermediate zeros), for all x G TL. Hence, the analogous 
results to Theorems 1, 2 and 3 for Hermitian matrices are true. 

Theorem 5 Let k G {!,..., g}, Xq G and d G be a unitary vector. Then 
there exists the directional derivative 

Crfc(xo,d). 
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the eigenvectors of the Hermitian 


Let us see that for any matrix B G 
matrix 

0 B 


H = 


B* 0 


are related with singular vectors of B. Let fc G {1,..., q} be such that ak{B) > 
0; we know that ak{B) is an eigenvalue of H. Let u G G be such 

that 

' u ' 


is an eigenvector of H associated with ak{B), i.e. 

0 B\ fu\ ( Bv 


Hi:)=akiB)r 


B* 0 \v 


n* , = (Tk{B) 

B u / \v 


Consequently, the nonzero vector 


is an eigenvector of H associated with ak{B) if and only if 

Bv = ak{B)u, 

B*u = ak{B)v. 

Either u 7 ^ 0, or u 7 ^ 0; were u 7 ^ 0 and u = 0, by (1), we would have u = 0, a 
contradiction; hence if u 7 ^ 0, then u 7 ^ 0. As the roles of u and v are symmetric, 
we deduce u 7 ^ 0 and u 7 ^ 0. Accordingly, 


( 1 ) 

( 2 ) 


is an eigenvector of H associated with ak{B) if and only if u and v are nonzero 
vectors that satisfy the conditions ( 1 ) and ( 2 ). 

Let Xo G be a fixed point, and let W G c(™+")x(™+") a unitary matrix 
that diagonalizes M{xq)-. 




W*M{xo)W = 


0 \ 


o-kixo) 


-CTkixo) 


(3) 


V 0 


-ai{xo)/ 


Suppose that 


CTi (xo) > • • • > (Jk-i^ (xo) > cTk-i^+i (xo) = ■■■ = (Jk (xo) 

= CTfe+i (xo) = • • • = (Tk+ik {xo) > cTk+j^+i (xo) > ■■■ > a-g (xo) > • • • > -cri(xo) 
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are the eigenvalues of M{xq), where ak{xo) is a multiple eigenvalue of multi¬ 
plicity Tk = ik + jk, ik being the number of eigenvalues equal to ak{xo) placed 
before the rank k + 1, and jk is the number of eigenvalues equal to ak(xo) situate 
after the rank k. 

Call W 2 to the (m-l-n) x matrix formed by the {k — ik + l)th,... ,{k+jk)th 
columns of the matrix W = [wi,..., Wm+n]- For each j G {1,..., m -I- n}, call 
Uj and Vj to the subvectors m x 1 and n x 1 , respectively, that constitute wj in 
this way 

= iu, 

As ||rcj|| = 1, then ||Mj|| < 1 and ||?;j|| < 1. This implies that Uj and Vj are not 
singular vectors. Because the before mentioned, for all j G {k—ik + 1, •. •, k+jk}. 


A{xo)vj = akixo)uj, 
A{xo)*Uj = akixo)vj. 
For each unitary vector d = (di,..., dp) G 

0 

.( 


(4) 

(5) 


F'{d) ■.= w;\Y,d, 


aA 

. 9xi 


define 

0 


Wo 


which is an rj, x Xk Hermitian matrix. Then, by Theorem 2, we have the next 
result. 

Theorem 6 For each unitary vector d = (di,..., dp)G 

a'k{xo,d) = /r,,(F'(d)), 

IJ,ik{F'{d)) being the ikth eigenvalue of the matrix F'{d) when we arrange the 
eigenvalues of this matrix in a decreasing order. 


To facilitate the writing let IT 2 be partitioned thus: 

W 2 = 

where 


'U 2 

1^2 


C 2 •— [Uk-ik + lj • ■ • ) ’^k+jk] ) F 2 [Vk-ik + l, • ■ • , Xk+jf.\ ) 

[/2 e e 

Corollary 7 For each unitary vector d = (di,..., dp)G we have 

a'k{xo,d) = 

yLik being the eigenvalue of 

that occupies the i^th place in the arrangement 

Ml > • • • > Mr, 

of the eigenvalues of this matrix. 
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Proof. Given that 


the matrix F'{d) is given by 


IP2 := 


U 2 
P 2 ’ 


vi=i 


dA* 

dxj 


0 |^(*o) 

(xo) 0 


1 

'U2 

J 

.^ 2 . 


= E^^- 

i=i 


\r*9A* , ,rr*9A. , 


dxj 


dx 



'U 2 


.^ 2 . 


f)A» GL r)A 

= E d, (^0) U2 + U*, E dj {xo) V2 


i=i 


i=i 


9xi 




dA 


= E dj ^ (^0) ^2 + I E Ty;: (^0) 


i=i 


i=i 


axi 


( 6 ) 


□ 

The sum of all singular values that coalesce with (Jk{xo) at xq is differentiable 
at xq. Even more it is true as we can see in the next theorem. 


Theorem 8 The function 


tk{x) := CT/c-ifc+i {x)-\ -h CTfe (x) H-h CTfe+j^ (x) 

is differentiable in a neighborhood V C fl of Xq. 

The neighborhood V is determined by the x G id sufficient close to xq in 
order that the inequalities 

cTk-i^x) > CTfc-i,+i{x) and CTfe+jJx) > ak+j^+i{x) 


hold. 


3 Function of Ikramov-Nazari 

With the notations of the paper [2], let (fi, ^ 2 , ^ 3 , ^ 4 ) € A S C”^". Define 

/A (6 + i 6 )A 

Q(6)'f2,C3,C4) := 0 A ^ 2 ! , u > 3. 

\0 0 A J 

Let f = (Ci, 6 , 6 .‘? 4 ) e be and 

fiO ■■=^3n-2{QiO)- 

Suppose that for a S the function / attains a local maximum ao '■= 
o’ 3 n- 2 (Q(^°)); let us assume also that erg > 0 and it is a multiple singular 
value of Q(^°). With the above notations, there are i 3 n -2 singular values before 
the place 3n — 2 + 1 and j 3 n -2 singular values after the place 3n — 2 equal 
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to a 3 n- 2 (Q(^^)) ■ For summarizing the notation let us rename p := i 3 n -2 and 
<7 := jzn- 2 - The multiplicity of ctq is m = p + g. Hence, 


(Q(^°)) > • • • > ^3n-2-p (Q(C°)) 

> 0'3r!,-2-p+l ((5(C°)) = • • • = CT3„_2 (Q(C°)) 

= Cr3„_2+1 (Q(C°)) = • • • = tT3„_2+g (Q(C°)) 

> 0-3„_2+q+l (Q(^°)) > • • ■ > Cr3„ (Q(C°)) • 

Here p > 1 and g > 0. The function 

^(0 •= Cr3ra-2-p+l {Q{0) -I-+ 0-3n-2+9 (0(0) 

is differentiable in a neighborhood of Also for each k G {1,..., 3n} and each 
unitary vector d S the function 

OkiO ■= o-fc(0(0) 

admits the directional derivative 


9k{e,d). 


Let us remark that the used notation implies 

/(0=53n-2(0, 


What relationship exists between the directional derivatives /'(O; d) and /'(Oi —d)7 
Given that / has a local maximum at Oj it follows that for all e S 


/'(0,e) 


lim 

h^0+ 


ne + he) - fie) 

h 


< 0 . 


Thus, < 0 and f'{^°,—d) < 0. When it will happen that f{^°,d) = 0 

for all d € such that ||d|j 2 = 1? By Theorem 6, f'{^^,d) is equal to pth 
eigenvalue Pp(d) of the m x m matrix 


with 


F'id) 





dQ* f cO) 
dfs > 


dQ 

dij 


■( 0 ) 

0 



U2 — ['U3n-2-p+l; ■ ■ ■ , U3n-2+q] 
V 2 = [u 3 „_ 2 -p+l, ■ ■ ■ , V 3 n- 2 +q] 


where Uj and Vj are the left and right singular vectors 


Q{^°)vj = aoUj 1 
Q{^°)*Uj = (ToVjj 


j = 3n — 2 — p + 1,..., 3n — 2 + g. 


and the eigenvalues of F'(d) are arranged in this way 


Pi (d) > • • • > pp (d) > • • • > pm{d) 


(7) 







Therefore, 


ne,d) = ^^pidy, 

by the analogous reason, /'(^°, —d) is equal to the pth eigenvalue of the Hermi- 
tian matrix F'{—d). But, pay attention, f'{^'^,—d) is not necessarily equal to 
fj,p{—d). In fact, if 

are the eigenvalues of F'{—d), then 

f{f,-d) = ap. 


As F'{—d) = —F'{d), it follows 

- Hmid)>---> -pip {d)>---> -pi(d) (8) 

are the eigenvalues of F'{—d)] whence, 

-d) = ap = -prn-{p-i)id). (9) 

Now it is necessary to analyze the relative positions of the indices p and m — 
(p-1)- 

If p < TO — (p — 1), then Pp{d) < 0; what implies 


0> Pp{d)>---> p™_(p_i) (d) > • • • > Pm{d); 

hence, 0 > /im-(p-i)(d); and so ap = —prn-(p-i){d) > 0 but Up = f'{C^,—d) < 
0. Thus, ap = 0; i.e. —d) = 0. Given that / has a local maximum at 5°, 

for all unitary vector e € we have 

/'(?°,e) = 0. 

Dubious case: If p > to — (p — 1), taking into account that Pp{d) < 0 and 
Pm-{p-i){d) > Ppid), it does not warrant that the sign of Prn-(p-i){d) be < 0. 


4 Average of singular values 

We know that the average of singular values of Q{^) that coalesce with the to- 
multiple singular value (T 3 „_ 2 (Q(C°)) at ^ is a differentiable function in a 

neighborhood of Thus we consider the differentiable function 


H{0 ■= tiO - mao; 


obviously, = 0- Hence, the point belongs to the level hypersurface of 

level 0 of the function H{^). Let 




m o. ^(, 0 ) ^(. 0 ) 



be the gradient of id(^) at Let d G such that 


Vid(^°)-d = 0, 
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where • denotes the ordinary scalar product in Then, by the chain rule, 

This implies 

0 = 53n-2-p+l(C°i d) + • • ■ + 53n-2(^°i d) + • • • + 5'3„_2+g(^°j d); 

if we consider the m x m Hermitian matrix F'{d), it means that the sum of its 
eigenvalues is zero: 

0 = /ri(d) + • • • + ^ip{d) + • • • + /im(d). 

When p = I, this is equivalent to say that cr 3„_2 (Q(^°)) is the first value of 
the chain of singular values equal to tro) then all the functions 

ff3n-2(0)ff3n-2+l(C)) ■ • ■ , 93n-2+q{0 

take the same value at and it is equal to (Tq- Moreover, all these functions 
have at a local maximum, because of 

fiO ■= 93n-2{0 > 93n-2+l{0 > ■ > 93n-2+q{0 ■ 

This implies that for all unitary d € 

Vfc = 3n-2, ...,3n-2 + ( 3 ', 5 fc(^°,d)< 0 ; 

therefore, ^i(d) < 0 ,..., pmid) < 0 , and, given that t(^) has a local maximum 
at and is differentiable at we have 

Vt(e°) = 0; 

whence = 0 and for all k = 3n — 2,..., 3n — 2 + 9 , d) = 0; in 

particular, d) = 0. This is proved because 0 = /ii (d) + • • • + pm (d); since 
yk,fj,k{d) < 0, we obtain Vfc,^fc(d) = 0; consequently, Vfc, ( 7 ^(^°,d) = 0. 

From now on let p be any integer from the range we are considering. Fur¬ 
thermore, suppose that for all fc = 3n — 2 —p-|-1,..., 3n — 2-|-q, all the functions 
9k{0 have a local maximum at Then for all unitary d € d) < 0. 

As t{^) has a local maximum at <'(^°,d) = 0; but 

d) = 53„_2 -p+i(?°) d) -I- • • • -I- g3„_2+5(C°) d); 
consequently, f'{C^,d) = 0 . 

When some of the functions gfc(C) have a local maximum at and any 
others have a local minimum at the analysis becomes more complicated and 
I do not obtain any conclusion. 

5 Remark 

In January 31, 2005,1 wrote an e-mail to J.B. Hiriart-Urruty asking him whether 
his results in [1] for real symmetric matrices kept true for complex Hermitian 
matrices. He forwarded my message to M. Torki [ 8 ], which answered me affirma¬ 
tively. Moreover, Torki told me that his results in [7] for second order directional 
derivatives and real symmetric matrices, were also true for the Hermitian case. 
Similar results were obtained by Lippert [4]. 
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